Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keycloak OIDC groups sync Issue: 7096 #9787

Closed
wants to merge 5 commits into from

Conversation

lme-nca
Copy link
Contributor

@lme-nca lme-nca commented Mar 19, 2024

Description
This pull request introduces group syncing for Keycloak (i.e. #7096 )

However the keycloak social auth provider (https://github.com/python-social-auth/social-core/blob/master/social_core/backends/keycloak.py) is really not a good implementation (it hardcodes the public key and string appends it......) and further more just bases itself of the BaseOAuth2 class and not the OICD class. Therefore in this pull request we replace the Keycloak provider with the plain: https://github.com/python-social-auth/social-core/blob/master/social_core/backends/open_id_connect.py this works perfectly with our local Keycloak instance on our Defectdojo fork.

As currently it seems "larger" pull requests are more difficult to merge, I have not "finished" this pull request, the documentation is not updated nor have I removed some of the "old" properties of the "keycloak" backend.

The question to the maintainers is if you will consider this change ? or if you think this is to large/breaking change....

If you are willing to merge it, I will clean it up further and incorporate any other improvements or suggestions you might have.

Test results

Ideally you extend the test suite in tests/ and dojo/unittests to cover the changed in this PR.
Alternatively, describe what you have and haven't tested.

Documentation

Documentation is not yet updated, if you are open to merging this then I would update the documentation

Checklist

This checklist is for your information.

  • Make sure to rebase your PR against the very latest dev.
  • Features/Changes should be submitted against the dev.
  • Bugfixes should be submitted against the bugfix branch.
  • Give a meaningful name to your PR, as it may end up being used in the release notes.
  • Your code is flake8 compliant.
  • Your code is python 3.11 compliant.
  • If this is a new feature and not a bug fix, you've included the proper documentation in the docs at https://github.com/DefectDojo/django-DefectDojo/tree/dev/docs as part of this PR.
  • Model changes must include the necessary migrations in the dojo/db_migrations folder.
  • Add applicable tests to the unit tests.
  • Add the proper label to categorize your PR.

Extra information

@github-actions github-actions bot added New Migration Adding a new migration file. Take care when merging. settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR ui labels Mar 19, 2024
Copy link

dryrunsecurity bot commented Mar 19, 2024

Hi there 👋, @DryRunSecurity here, below is a summary of our analysis and findings.

DryRun Security Status Findings
Configured Codepaths Analyzer 10 findings
Sensitive Files Analyzer 0 findings
AppSec Analyzer 0 findings
Authn/Authz Analyzer 12 findings
Secrets Analyzer 0 findings

Note

🔴 Risk threshold exceeded. Adding a reviewer if one is configured in .dryrunsecurity.yaml.

notification list: @mtesauro @grendel513

Change Summary (click to expand)

The following is a summary of changes in this pull request made by me, your security buddy 🤖. Note that this summary is auto-generated and not meant to be a definitive list of security issues but rather a helpful summary from a security perspective.

Summary:

The code changes in this pull request focus on enhancing the security and authentication mechanisms of the DefectDojo application. The key changes include:

  1. Keycloak and OIDC Integration: The application is adding support for Keycloak and OpenID Connect (OIDC) as authentication providers, allowing users to authenticate using these identity management systems. This improves the application's flexibility and enables integration with a wider range of identity providers.

  2. Group Membership Management: The changes introduce functionality to synchronize user groups between the identity providers (Keycloak, Azure AD) and the DefectDojo application. This ensures proper access control and permissions management within the application.

  3. Security Headers and Configurations: The application is setting various security-related headers and configurations, such as X-Content-Type-Options, HttpOnly and Secure flags for cookies, and HSTS settings. These measures help improve the overall security posture of the application.

  4. Logging and Monitoring: The application is enhancing its logging and monitoring capabilities, including the use of Django metrics for better observability and troubleshooting.

  5. Deduplication and Hashcode Configuration: The changes include extensive configuration options for deduplicating findings, allowing the application to handle this process more effectively.

  6. JIRA Integration: The application is improving its integration with the JIRA issue tracking system, including the ability to configure SSL verification and supported issue types.

  7. File Uploads: The application is setting a maximum file size limit and defining a list of acceptable file types for scan file uploads, which helps mitigate potential security risks associated with file uploads.

Files Changed:

  1. dojo/group/utils.py: The changes focus on the handling of group creation and membership, ensuring that the current user is only added as the owner of a newly created group if the application is not configured to retrieve group information from Azure AD or Keycloak.
  2. dojo/context_processors.py: The changes add support for Keycloak authentication, including the introduction of new configuration variables and a Keycloak login button.
  3. dojo/db_migrations/0213_alter_dojo_group_social_provider.py: This migration modifies the social_provider field of the dojo_group model, making it nullable and limiting the allowed values to 'AzureAD', 'Remote', and 'Keycloak'.
  4. docs/content/en/integrations/social-authentication.md: The changes document the integration of various OAuth2 and OIDC providers, including Keycloak, with the DefectDojo application.
  5. dojo/models.py: The changes add a new choice, 'Keycloak', to the SOCIAL_CHOICES field in the Dojo_Group model, indicating support for Keycloak as a social authentication provider.
  6. dojo/settings/.settings.dist.py.sha256sum: The changes update the SHA-256 hash value for the dojo/settings/.settings.dist.py file, which is used for integrity verification.
  7. dojo/user/views.py: The changes update the social authentication provider from "keycloak" to "oidc" (OpenID Connect) in the login_view function.
  8. dojo/pipeline.py: The changes focus on handling user authentication, group membership, and product access management, integrating with various identity providers.
  9. dojo/templates/dojo/login.html: The changes update the URL for the Keycloak login button to use the more generic OIDC integration.
  10. dojo/settings/settings.dist.py: The changes include extensive security-related configurations, such as authentication and authorization settings, security headers, logging and monitoring, Celery tasks, deduplication, and file upload restrictions.

Overall, these changes demonstrate a strong focus on enhancing the security and authentication capabilities of the DefectDojo application, which is crucial for maintaining the integrity and confidentiality of the data managed by the tool.

Powered by DryRun Security

@mtesauro
Copy link
Contributor

@lme-nca I've re-kicked the tests but even if they all pass, you'll need to fix the Ruff linter issue before we can start the approval process.

@Maffooch
Copy link
Contributor

@lme-nca this looks like the proposal is to use a generic OID backend with some key cloak references mixed in. If we are to generalize on OIDC, I think it should be done all the way.

This proposal raises a question: do we want to support a generic OIDC mechanism? It seems like OAuth is more widely used at this point, and would add more value. @mtesauro what do you think?

@lme-nca
Copy link
Contributor Author

lme-nca commented Mar 25, 2024

Hi @Maffooch & @mtesauro,

Thanks for looking at the pull request.

Currently defectdojo is kind of already using OIDC not just Oauth in your keycloak implementation, see defectdojos own documentation on how to set up the client in Keycloak: https://defectdojo.github.io/django-DefectDojo/integrations/social-authentication/#configure-keycloak Keycloak always provides oidc on top of Oauth, like just about any other authentication provider. Furthermore, the current keycloak provider is unusable as it hardcodes the public key, making key rotation by the provider impossible.....

Another example is Defectdojo's "Auth0" integration, which is based on https://github.com/python-social-auth/social-core/blob/master/social_core/backends/auth0.py which also reads the "id_token" value, which is a OIDC only concept, even though the class only inherits from the "BaseOAuth2" baseclass. It should probably instead also inherit from https://github.com/python-social-auth/social-core/blob/master/social_core/backends/open_id_connect.py The same goes for OCTA which uses the OIDC well known endpoint: https://github.com/python-social-auth/social-core/blob/master/social_core/backends/okta.py

The fact is that nearly all Authentication providers use a mix of Oauth and OIDC protocol elements to provider their services and a lot of the providers in https://github.com/python-social-auth/social-core/tree/master are not distinguishing this correctly.

@Maffooch

The problem with generalizing to OID is that this pull request relies on keycloak specific groups and mapper mechanisms to sync groups from keycloak to defectdojo (i.e. it requires the use of https://www.keycloak.org/docs-api/21.1.2/javadocs/org/keycloak/protocol/oidc/mappers/GroupMembershipMapper.html ). Now other providers probably have very similar mechanisms but if they work the same depends heavily on their configuration..... Therefore I would suggest that from a user point of view this is all Keycloak specific (that this uses the generic OIDC connect backend is irrelevant).

Therefore I propose to keep the configuration from Defectdojo side "keycloak" specific, while under water using the OIDC provider of social core to properly integrate with keycloak.

alternatively: the authentication part of keycloak, could be made generic "OIDC" and the configurations of the group syncing would then be "keycloak" specific named configuration settings. Then the documentation would reflect that the usage of keycloak as an authentication provider would just be 1 example of using a OIDC compatible provider with the option to enable keycloak specific group syncing....

@mtesauro
Copy link
Contributor

@lme-nca I'm not opposed to this change in the keycloak integration - I get the points you're making about these being Keycloak-specific groups.

However, to move forward, the tests would have to be green both Ruff and the various unit-tests.

@lme-nca lme-nca force-pushed the keycloak_oidc_groups_sync branch 4 times, most recently from 3676e63 to 9f548bd Compare April 23, 2024 15:33
@github-actions github-actions bot added the docs label Apr 23, 2024
@lme-nca lme-nca force-pushed the keycloak_oidc_groups_sync branch 3 times, most recently from 62334b8 to 9624f52 Compare April 24, 2024 06:37
@lme-nca
Copy link
Contributor Author

lme-nca commented Apr 26, 2024

@lme-nca I'm not opposed to this change in the keycloak integration - I get the points you're making about these being Keycloak-specific groups.

However, to move forward, the tests would have to be green both Ruff and the various unit-tests.

@mtesauro Ok the tests are green now, I also made a note in the documentation (docs/content/en/integrations/social-authentication.md) as this should still be updated. Let me know if you agree with the technical part of the Pull request and then ill update the documentation to match the new process.

Copy link
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@lme-nca lme-nca force-pushed the keycloak_oidc_groups_sync branch from 1a1ec7b to 695fab4 Compare May 21, 2024 20:36
Copy link
Contributor

Conflicts have been resolved. A maintainer will review the pull request shortly.

@lme-nca lme-nca force-pushed the keycloak_oidc_groups_sync branch 4 times, most recently from 12b8034 to 526c1d4 Compare May 24, 2024 08:09
@lme-nca lme-nca force-pushed the keycloak_oidc_groups_sync branch from e4f162b to 78dbb91 Compare May 29, 2024 07:00
@lme-nca lme-nca force-pushed the keycloak_oidc_groups_sync branch from 78dbb91 to 679a167 Compare May 30, 2024 12:55
@lme-nca
Copy link
Contributor Author

lme-nca commented May 30, 2024

@mtesauro @Maffooch All tests pass, documentation has been updated. Would really like to merge this to avoid further conflicts down the road.....

Copy link
Contributor

github-actions bot commented Jun 3, 2024

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@Maffooch
Copy link
Contributor

Thank you for contributing to DefectDojo! We have a mantra around here with regards to merging a pull request: “A no is just for today, but a yes is forever”. With that said, we must say no to this PR for today because it has gone stale

@Maffooch Maffooch closed this Nov 15, 2024
@lme-nca
Copy link
Contributor Author

lme-nca commented Nov 20, 2024

@Maffooch @mtesauro : So we are just happy to have a broken and insecure Keycloak integration in Defectdojo ??? I really would like to fix it, but for this I need some kind of Project support to get this pull request approved ?

@mtesauro
Copy link
Contributor

@lme-nca

So we are just happy to have a broken and insecure Keycloak integration in Defectdojo ???

No, of course not. I don't have Keycloak nor any experience with it so I'm not in a place to say if there are deficiencies in the support for Keycloak we get from the social-auth Python module we're using.

I really would like to fix it, but for this I need some kind of Project support to get this pull request approved ?

We'd really like someone in the community with access to Keycloak to help maintain that SSO provider. @Maffooch wrote "because it has gone stale" which it had when he closed this PR.

One of the things we're starting to use to keep the PR list clean is to look at the last time an update happened to a PR with a conflict. There's been no updates to this PR since a merge conflict happened back on June 3rd. I think if you ask most code maintainers if a PR which a merge conflict and no updates for ~4 months was stale, they'd say yes. That's what we did here.

You're welcome to re-open this PR or start a new one - @Maffooch did say "A no is just for today, but a yes is forever" which seems fair for a feature that requires access to a 3rd party system which is non-trivial to setup and maintain.

You might also look at this currently open PR - #10614 that is adding generic OIDC login which might accomplish what you were looking for here.

Cheers!

@lme-nca
Copy link
Contributor Author

lme-nca commented Nov 21, 2024

@mtesauro the conflict happens because of the HASH check (or in my case database change) which happens every single time another change gets merged (that changes the hash or db)... it would be easier to review the code, approve it and then when we are ready to merge fix this last "conflict" which is not a real conflict but an artificial barrier introduced that is actively stopping people from contributing as these pull request simply get ignored/not reviewed afterwards...

(look here, he also went trough the useless effort: f97900e ) sadly since then there is a new conflict..... and now he has to jump through hoops again....

Honestly, there is a great community of people who would like to contribute, but everyones time is valuable and this process is disrespectful of the time that people put into these pull requests.

(EDIT)
I see that you recognise this too now: #11299

@mtesauro
Copy link
Contributor

@lme-nca Yes, you found what I was going to point out.

We're aware that an effort to try to keep people deploying and using DefectDojo in a incorrect way has had the side effect of making it harder for the community to make contributions since they are frequently getting tripped up by that sha256sum. And, we're working through the best way to remove that contribution burden. If you're still interested in restarting this work and want to wait till that's merged, I completely understand that position.

BTW, I don't believe the script that Maffooch used to find the stale PRs was clever enough to make the distinction between sha256 conflicts and other merge conflicts though I've not seen it myself. TBH, I'm not sure if GH's API gives you those detail as I've never looked at that part of their API. Once #11299 is merged, that lack of distinction should no longer matter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
conflicts-detected docs New Migration Adding a new migration file. Take care when merging. settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR ui
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants